<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Image Denoising</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d0/d07/tutorial_py_table_of_contents_photo.html">Computational Photography</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Image Denoising </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter,</p>
<ul>
<li>You will learn about Non-local Means Denoising algorithm to remove noise in the image.</li>
<li>You will see different functions like <b><a class="el" href="../../d1/d79/group__photo__denoise.html#ga4c6b0031f56ea3f98f768881279ffe93" title="Perform image denoising using Non-local Means Denoising algorithm http://www.ipol.im/pub/algo/bcm_non_local_means_denoising/ with several computational optimizations. Noise expected to be a gaussian white noise. ">cv.fastNlMeansDenoising()</a></b>, <b><a class="el" href="../../d1/d79/group__photo__denoise.html#ga03aa4189fc3e31dafd638d90de335617" title="Modification of fastNlMeansDenoising function for colored images. ">cv.fastNlMeansDenoisingColored()</a></b> etc.</li>
</ul>
<h2>Theory </h2>
<p>In earlier chapters, we have seen many image smoothing techniques like Gaussian Blurring, Median Blurring etc and they were good to some extent in removing small quantities of noise. In those techniques, we took a small neighbourhood around a pixel and did some operations like gaussian weighted average, median of the values etc to replace the central element. In short, noise removal at a pixel was local to its neighbourhood.</p>
<p>There is a property of noise. Noise is generally considered to be a random variable with zero mean. Consider a noisy pixel, \(p = p_0 + n\) where \(p_0\) is the true value of pixel and \(n\) is the noise in that pixel. You can take large number of same pixels (say \(N\)) from different images and computes their average. Ideally, you should get \(p = p_0\) since mean of noise is zero.</p>
<p>You can verify it yourself by a simple setup. Hold a static camera to a certain location for a couple of seconds. This will give you plenty of frames, or a lot of images of the same scene. Then write a piece of code to find the average of all the frames in the video (This should be too simple for you now ). Compare the final result and first frame. You can see reduction in noise. Unfortunately this simple method is not robust to camera and scene motions. Also often there is only one noisy image available.</p>
<p>So idea is simple, we need a set of similar images to average out the noise. Consider a small window (say 5x5 window) in the image. Chance is large that the same patch may be somewhere else in the image. Sometimes in a small neighbourhood around it. What about using these similar patches together and find their average? For that particular window, that is fine. See an example image below:</p>
<div class="image">
<img src="../../nlm_patch.jpg" alt="nlm_patch.jpg"/>
<div class="caption">
image</div></div>
<p> The blue patches in the image looks the similar. Green patches looks similar. So we take a pixel, take small window around it, search for similar windows in the image, average all the windows and replace the pixel with the result we got. This method is Non-Local Means Denoising. It takes more time compared to blurring techniques we saw earlier, but its result is very good. More details and online demo can be found at first link in additional resources.</p>
<p>For color images, image is converted to CIELAB colorspace and then it separately denoise L and AB components.</p>
<h2>Image Denoising in OpenCV </h2>
<p>OpenCV provides four variations of this technique.</p>
<ol type="1">
<li><b><a class="el" href="../../d1/d79/group__photo__denoise.html#ga4c6b0031f56ea3f98f768881279ffe93" title="Perform image denoising using Non-local Means Denoising algorithm http://www.ipol.im/pub/algo/bcm_non_local_means_denoising/ with several computational optimizations. Noise expected to be a gaussian white noise. ">cv.fastNlMeansDenoising()</a></b> - works with a single grayscale images</li>
<li><b><a class="el" href="../../d1/d79/group__photo__denoise.html#ga03aa4189fc3e31dafd638d90de335617" title="Modification of fastNlMeansDenoising function for colored images. ">cv.fastNlMeansDenoisingColored()</a></b> - works with a color image.</li>
<li><b><a class="el" href="../../d1/d79/group__photo__denoise.html#gaf4421bf068c4d632ea7f0aa38e0bf172" title="Modification of fastNlMeansDenoising function for images sequence where consecutive images have been ...">cv.fastNlMeansDenoisingMulti()</a></b> - works with image sequence captured in short period of time (grayscale images)</li>
<li><b><a class="el" href="../../d1/d79/group__photo__denoise.html#gaa501e71f52fb2dc17ff8ca5e7d2d3619" title="Modification of fastNlMeansDenoisingMulti function for colored images sequences. ">cv.fastNlMeansDenoisingColoredMulti()</a></b> - same as above, but for color images.</li>
</ol>
<p>Common arguments are:</p><ul>
<li>h : parameter deciding filter strength. Higher h value removes noise better, but removes details of image also. (10 is ok)</li>
<li>hForColorComponents : same as h, but for color images only. (normally same as h)</li>
<li>templateWindowSize : should be odd. (recommended 7)</li>
<li>searchWindowSize : should be odd. (recommended 21)</li>
</ul>
<p>Please visit first link in additional resources for more details on these parameters.</p>
<p>We will demonstrate 2 and 3 here. Rest is left for you.</p>
<h3>1. <a class="el" href="../../d1/d79/group__photo__denoise.html#ga03aa4189fc3e31dafd638d90de335617" title="Modification of fastNlMeansDenoising function for colored images. ">cv.fastNlMeansDenoisingColored()</a></h3>
<p>As mentioned above it is used to remove noise from color images. (Noise is expected to be gaussian). See the example below: </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">from</span> matplotlib <span class="keyword">import</span> pyplot <span class="keyword">as</span> plt</div><div class="line"></div><div class="line">img = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;die.png&#39;</span>)</div><div class="line"></div><div class="line">dst = <a class="code" href="../../d1/d79/group__photo__denoise.html#ga03aa4189fc3e31dafd638d90de335617">cv.fastNlMeansDenoisingColored</a>(img,<span class="keywordtype">None</span>,10,10,7,21)</div><div class="line"></div><div class="line">plt.subplot(121),plt.imshow(img)</div><div class="line">plt.subplot(122),plt.imshow(dst)</div><div class="line">plt.show()</div></div><!-- fragment --><p> Below is a zoomed version of result. My input image has a gaussian noise of \(\sigma = 25\). See the result:</p>
<div class="image">
<img src="../../nlm_result1.jpg" alt="nlm_result1.jpg"/>
<div class="caption">
image</div></div>
 <h3>2. <a class="el" href="../../d1/d79/group__photo__denoise.html#gaf4421bf068c4d632ea7f0aa38e0bf172" title="Modification of fastNlMeansDenoising function for images sequence where consecutive images have been ...">cv.fastNlMeansDenoisingMulti()</a></h3>
<p>Now we will apply the same method to a video. The first argument is the list of noisy frames. Second argument imgToDenoiseIndex specifies which frame we need to denoise, for that we pass the index of frame in our input list. Third is the temporalWindowSize which specifies the number of nearby frames to be used for denoising. It should be odd. In that case, a total of temporalWindowSize frames are used where central frame is the frame to be denoised. For example, you passed a list of 5 frames as input. Let imgToDenoiseIndex = 2 and temporalWindowSize = 3. Then frame-1, frame-2 and frame-3 are used to denoise frame-2. Let's see an example. </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"><span class="keyword">from</span> matplotlib <span class="keyword">import</span> pyplot <span class="keyword">as</span> plt</div><div class="line"></div><div class="line">cap = <a class="code" href="../../d8/dfe/classcv_1_1VideoCapture.html">cv.VideoCapture</a>(<span class="stringliteral">&#39;vtest.avi&#39;</span>)</div><div class="line"></div><div class="line"><span class="comment"># create a list of first 5 frames</span></div><div class="line">img = [cap.read()[1] <span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(5)]</div><div class="line"></div><div class="line"><span class="comment"># convert all to grayscale</span></div><div class="line">gray = [<a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(i, cv.COLOR_BGR2GRAY) <span class="keywordflow">for</span> i <span class="keywordflow">in</span> img]</div><div class="line"></div><div class="line"><span class="comment"># convert all to float64</span></div><div class="line">gray = [np.float64(i) <span class="keywordflow">for</span> i <span class="keywordflow">in</span> gray]</div><div class="line"></div><div class="line"><span class="comment"># create a noise of variance 25</span></div><div class="line">noise = np.random.randn(*gray[1].shape)*10</div><div class="line"></div><div class="line"><span class="comment"># Add this noise to images</span></div><div class="line">noisy = [i+noise <span class="keywordflow">for</span> i <span class="keywordflow">in</span> gray]</div><div class="line"></div><div class="line"><span class="comment"># Convert back to uint8</span></div><div class="line">noisy = [np.uint8(np.clip(i,0,255)) <span class="keywordflow">for</span> i <span class="keywordflow">in</span> noisy]</div><div class="line"></div><div class="line"><span class="comment"># Denoise 3rd frame considering all the 5 frames</span></div><div class="line">dst = <a class="code" href="../../d1/d79/group__photo__denoise.html#ga723ffde1969430fede9241402e198151">cv.fastNlMeansDenoisingMulti</a>(noisy, 2, 5, <span class="keywordtype">None</span>, 4, 7, 35)</div><div class="line"></div><div class="line">plt.subplot(131),plt.imshow(gray[2],<span class="stringliteral">&#39;gray&#39;</span>)</div><div class="line">plt.subplot(132),plt.imshow(noisy[2],<span class="stringliteral">&#39;gray&#39;</span>)</div><div class="line">plt.subplot(133),plt.imshow(dst,<span class="stringliteral">&#39;gray&#39;</span>)</div><div class="line">plt.show()</div></div><!-- fragment --><p> Below image shows a zoomed version of the result we got:</p>
<div class="image">
<img src="../../nlm_multi.jpg" alt="nlm_multi.jpg"/>
<div class="caption">
image</div></div>
<p> It takes considerable amount of time for computation. In the result, first image is the original frame, second is the noisy one, third is the denoised image.</p>
<h2>Additional Resources </h2>
<ol type="1">
<li><a href="http://www.ipol.im/pub/art/2011/bcm_nlm/">http://www.ipol.im/pub/art/2011/bcm_nlm/</a> (It has the details, online demo etc. Highly recommended to visit. Our test image is generated from this link)</li>
<li><a href="https://www.coursera.org/course/images">Online course at coursera</a> (First image taken from here)</li>
</ol>
<h2>Exercises </h2>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
